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Abstract 

The  General  Services  Administration  (GSA)  Heartland  Region  is  implementing  a  best 
value  process  (which  minimizes  time  and  cost  deviations  98%  of  the  time)  and  which 
minimizes  the  need  for  client’s  decision  making,  transfers  the  risk  and  control  of  a  project  to 
the  vendor,  and  forces  the  vendor  to  manage  and  minimize  the  non-technical  risk  that  the 
vendor  does  not  control.  The  Performance  Information  Risk  Management  System  (PIRMS) 
has  been  tested  by  the  US  Army  Medical  Command  and  has  minimized  over  50%  of  client 
project  management  and  risk  management  transactions,  and  also  minimized  cost  and  time 
deviations  by  as  much  as  70%.  The  new  paradigm  uses  Deming’s  concept  of  managing 
and  minimizing  the  project  deviation  instead  of  meeting  minimum  standards.  The  system 
forces  the  client’s  representatives  to  do  quality  assurance,  and  the  vendor  to  do  quality 
control.  The  mechanism  used  is  a  risk  management  plan  and  a  weekly  risk  report  that 
creates  transparency  between  buyer  and  vendor.  The  system  can  minimize  up  to  90%  of 
the  government’s  transactions  and  activities.  The  system  is  a  new  paradigm  for  government 
systems. 

Keywords:  Best  value  procurement,  minimized  government  management,  high 
vendor  performance,  and  measured  environment 

Introduction 

The  General  Services  Administration  (GSA)  is  the  largest  buyer  of  non-military 
services  in  the  United  States.  It  is  a  large  management  based  organization.  An  Achilles 
heel  for  any  large  organization  is  the  number  of  layers  of  management,  the  large  number  of 
managers  and  subject  matter  experts  (SME)  and  the  practice  of  managing,  directing  and 
controlling  vendors/contractors  who  are  supposed  to  be  experts  at  what  they  do. 

The  current  status  of  most  projects  in  the  GSA  is  where  vendors  (architect/ 
engineers  and  contractors)  continually  rely  on  being  managed,  directed,  and  controlled  by 
GSA  project  managers  and  contracting  officers.  To  get  a  quality  set  of  construction 
documents,  the  government  project  managers  complete  extensive  quality  control  reviews  of 
the  A/E’s  construction  documents,  once  the  sole  responsibility  of  the  A/E  design  firms.  GSA 
personnel  (CORs,  PMs,  and  COs)  continually  manage,  direct,  and  control  the  contractors  in 
construction.  Control  and  risk  are  not  transferred  to  the  vendors,  thus  making  it  difficult  to 
hold  vendors  accountable  for  deviations.  Projects  incur  change  orders  due  to  design 
deficiencies.  Projects  are  not  being  completed  in  a  timely  manner  and  the  actual  close-out 

ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  610 
NAVAL  POSTGRADUATE  SCHOOL 


of  projects  could  take  between  1  to  4  years.  GSA  processes  and  requirements  are 
continually  being  developed  and  expanded  at  various  levels  both  nationally  and  regionally  in 
an  attempt  to  increase  the  performance  of  the  vendors. 

Upper  level  management  in  the  GSA  has  struggled  with  implementing  a  sustainable, 
useable,  and  accurate  system  that  measures  the  performance  of  their  vendors  and  project 
managers.  Shrinking  budgets,  increased  workload  requirements,  and  the  increased  need 
for  project  managers  to  manage,  direct,  and  control  vendors,  make  the  updating,  collection, 
and  analysis  of  performance  information  very  difficult.  The  GSA  has  been  exposed  to  many 
management  measurement  systems  and  philosophies  (Alsup,  2010;  Topi,  2010): 

1 .  Quality  Management  Circle  (part  of  TQM) 

2.  TQM  (total  quality  management)  (early  1990s) 

3.  eTMP  (electronic  transaction  management  playbook)  (2006) 

4.  TMP  (transaction  management  playbook)  (2006) 

5.  HCAM  (included  the  following  TMP,  OMP,  LCP,  &  AMP)  (2005) 

6.  OMP  (occupancy  management  playbook) 

7.  LCP  (large  construction  playbook) 

8.  AMP  (account  management  playbook) 

However,  the  strategic  objective  of  increasing  performance  and  value  of  vendor 
services  and  measurement  of  the  performance  remains  elusive. 

Problem 

The  stubbornness  of  the  problem  of  the  GSA’s  inability  to  sustain  performance 
measurements  in  a  timely  fashion  and  increase  vendor  performance  may  be  a  systems 
problem  and  not  a  GSA  unique  management/leadership  problem.  The  current  GSA  system 
forces  the  project  managers  to  document,  maintain,  and  report  the  performance  information. 
Because  of  the  current  project  manager/vendor  relationships  and  their  heavy  workload, 
project  managers  may  not  be  motivated  to  accurately  and  consistently  document  the 
performance  information.  The  current  system  of  delivery  has  the  following  suboptimal 
characteristics: 

1 .  The  GSA  project  managers  are  required  to  manage,  direct,  and  control  the 
vendors. 

2.  There  is  no  transfer  of  risk  or  control  to  the  vendors. 

3.  The  relationship  between  the  vendors  and  the  project  managers  may  dilute 
accountability. 

4.  The  current  delivery  system  does  not  motivate  vendors  to  preplan  and 
manage  and  minimize  the  risk  that  they  do  not  control  (think  in  the  best 
interest  of  the  client). 

Hypothesis 

Deming  (1982)  identifies  the  GSA  problem  as  a  systems  problem.  The  authors 
propose  that  the  system  may  be  stabilized  but  not  meeting  the  expectations  of  the  GSA’s 
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upper  level  management.  Increasing  effort  to  optimize  performance  in  a  stable  environment 
may  not  be  successful.  The  system  must  be  changed  to  increase  performance,  and  have  a 
sustainable  performance  measurement  system  that  results  in  an  increase  in  vendor 
performance. 

Methodology 

The  authors  propose  that  the  GSA  find  a  new  system  that  has  the  following 
characteristics: 

1 .  Run  by  a  core  team  of  systems  managers  who  understand  a  performance 
information  based  system. 

2.  Selects  vendors  by  their  capability  to  understand  a  project  and  manage  and 
minimize  deviations  that  are  caused  by  sources  outside  of  their  control. 

3.  Transfer  the  risk  and  control  to  vendors,  who  by  contract  must  preplan, 
manage  and  minimize  risk  that  they  do  not  control,  and  manage  the  risk  of 
their  projects  by  measuring  and  minimizing  cost  and  time  deviations. 

4.  Measure  the  vendors’  performance. 

5.  Vendors  are  held  accountable  for  all  deviations  unless  they  can  dominantly 
document  the  source  of  the  risk  and  how  they  attempted  to  manage  and 
minimize  the  risks. 

6.  System  has  past  performance  of  success  in  other  government  organizations. 

Search  for  Performance  Measurement  Systems  for 
Vendors/Organizations 

The  GSA  team  composed  of  a  project  manager  and  a  procurement  officer  proposed 
the  above  plan  to  their  division  manager  in  the  heartland  region  located  in  Kansas  City. 

They  proposed  using  a  best  value  approach  to  solicitation  to  procure  the  services  of  an 
organization  that  could: 

1 .  Provide  a  system  that  meets  the  requirements  of  the  new  system. 

2.  Show  documented  evidence  that  the  system  measured  performance  and 
resulted  in  a  dominant  increase  of  performance. 

3.  Show  capability  to  educate  and  train  GSA  personnel  to  be  able  to  understand 
the  new  system  and  run  the  system. 

The  solicitation  was  posted  on  August  15,  2009.  There  were  only  three  proposers. 
The  GSA  identified  the  Performance  Information  Procurement  System  (PIPS)  as  the  only 
system  that  could  meet  the  new  requirements.  It  was  the  only  system  that: 

Had  documented  performance  over  15  years. 

Minimized  the  amount  of  management,  direction,  and  control. 

Transferred  risk  and  accountability  to  the  vendor. 

Measured  the  vendor  and  all  other  participants  who  interfaced  with  the 
vendor. 
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5.  Provided  a  mechanism  whereby  the  vendor  managed  the  risk  that  they  did 
not  control. 

6.  Minimized  the  transactions  (meetings,  emails,  and  telecom.) 

No  other  vendor  options  would  accept  responsibility  for  ensuring  the  performance 
measurements  on  all  participants,  a  resulting  increase  in  performance  and  value,  and 
minimized  management,  direction  and  control.  The  other  vendor  options  proposed  to  put  a 
performance  measurement  system  in  place,  but  would  take  no  responsibility  for  the 
successful  implementation  or  the  impact  on  performance  of  the  organization  and  the 
vendors.  The  best  value  PIPS  system  dominantly  differentiated  itself  as  the  only  option  with 
the  potential  of  disruptive  change  required  to  employ  a  sustainable  measurement  system 
and  result  in  a  dominant  increase  in  performance  and  value. 

Industry  Analysis 

The  construction  industry  structure  (CIS)  model  (Figure  1)  (Kashiwagi,  2009) 
identifies  the  difference  between  what  the  GSA  and  other  owners  currently  employ,  and 
what  is  needed  to  employ  a  sustainable  performance  measurement  system  that  has 
accompanying  increase  in  performance  and  value. 


III.  Neaotlated-Bid 

Owner  selects  vendor 

Negotiates  with  vendor 

Vendor  performs 

II.  Best  Value 

Performance  and  price 

Contractor  creates 
baseline  plan 

Contractor  justifies  and 
measures  deviations 

Contractor  is  technical 
expert 

IV.  Unstable  Market 

1.  Price  Based 

Specifications,  standards 
and  qualification  based 

Management,  direction, 
control  &  inspection  by 
client's  professional 

Client's  professional  is 
technical  expert/decision 
maker 

Low  Competition  High 


Construction  Industry  Structure 

(Kashiwagi,  2010) 

The  CIS  identified  the  current  GSA  system  as  a  price  based  system  regardless  of  the 
perception  of  not  awarding  projects  based  on  price  (Sullivan,  2005).  The  Price  Based 
System  (Quadrant  I),  has  the  following  characteristic: 


1 .  Owner  representative  attempts  to  direct  and  control  the  vendor. 

2.  Deductive  logic  assumes  that  the  owner  would  only  hire  a  vendor  who  knew 
what  they  were  doing  (expert). 

3.  Someone  who  should  know  less  about  what  is  being  done  is  directing 
someone  who  is  an  expert  in  what  is  being  delivered. 

In  the  best  value  system,  the  client  identifies  their  intent,  but  asks  the  industry 
(vendors)  who  can  deliver  the  best  value  that  meets  the  intent  of  the  owner?  This  system 
has  the  following  characteristics: 

Vendors  identify  what  they  do,  and  what  it  costs. 
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2.  Client  selects  the  best  value  based  on  performance  and  price  (value). 

3.  Vendors  preplan,  manage  and  minimize  risk  they  don’t  control.  (They  have  no 
technical  risk;  they  are  experts.) 

4.  Clients  do  quality  assurance  (ensure  that  the  vendor  has  quality  control  and 
risk  management  systems). 

■ 

The  price  based  system  is: 

1 .  Inefficient. 

2.  Requires  more  people. 

3.  Has  more  confusion. 

4.  Subjective. 

5.  Higher  flow  of  information  required  between  parties. 

6.  If  performance  measures  are  kept,  they  are  very  subjective  and  potentially 

could  be  construed  as  biased.  There  are  more  measurements  that  require  an 
expert  to  decide  what  is  good  performance. 

7.  Requires  people  to  partner. 

8.  Relationships  are  important  between  vendors  and  the  client’s  PMs. 

9.  Adversarial. 

10.  Because  minimum  requirements  are  used,  performance  will  be  in  decline. 

1 1 .  Not  accountable. 

12.  Has  high  risk. 

13.  Low  performance  (on  time,  on  budget,  meet  client’s  expectations.) 

The  best  value  system: 

Is  more  efficient. 

Requires  less  people. 

Is  measured  with  few  and  simple  measurements. 

Is  transparent. 

Uses  alignment  of  resources. 

Has  minimized  flow  of  information. 

Minimizes  partnering  exercises. 

Managed  by  minimizing  deviations. 

Has  continuous  improvement. 

Has  a  high  level  of  accountability. 

Has  high  performance  and  low  risk. 
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12.  Minimizes  transactions. 


Difference  Between  the  Two  Systems 


The  difference  between  systems  is  that  the  price  based  vendor  is  reactive,  managed, 
controlled  and  directed  by  the  client/buyer;  the  best  value  vendor  is  proactive,  preplans, 
manages  and  minimizes  risk  that  they  do  not  control,  measures  their  performance,  and 
manages  their  project  by  minimizing  deviations.  In  the  price-based  sector,  the  client  directs 
the  vendor  using  minimum  requirements;  the  vendors  transform  the  minimum  into  a 
maximum,  and  drive  the  performance  the  opposite  direction  (Figure  2). 
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Figure  1. 


Min/Max  Dilemma 
(Kashiwagi,  2010) 


Figure  2. 


Price-Based  Award 

(Kashiwagi,  2010) 


The  high  performance  vendor  (high  performance  and  low  risk)  can  see  a  project  from 
beginning  to  end.  They  identify  risk  that  they  do  not  control,  and  plan  solutions  to  minimize 
the  risk.  The  low  performer  (high  risk)  prices  only  what  they  are  directed  to  price.  The 
overall  effect  of  the  client  directing  the  vendors  is  the  following  (Figure  3): 


1 .  High  performers  become  reactive  instead  of  proactive.  They  are  told  to  price 
only  what  is  directed,  regardless  of  completeness,  correctness,  or  whether  it 
is  doable. 


2.  They  are  directed  to  assume  that  the  directions  are  perfect. 

3.  Therefore,  they  are  directed  to  give  the  lowest  possible  price,  assuming 
nothing  goes  wrong. 


Figure  4  shows  the  business  approach  to  Quadrant  I  Price  Based  system.  The  high 
performers,  who  get  paid  more  for  their  expertise,  get  sent  to  the  price  based  system,  where 
they  are  directed  and  controlled  by  someone  who  is  not  an  expert.  The  confusing 
environment  results  in  lower  production  of  the  high  performer.  They  become  out  of 
alignment,  overpriced,  and  leave  the  environment.  A  more  damaging  result  of  the  system  is 
that  the  less  experienced  are  not  motivated  to  become  like  the  highly  trained. 
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Figure  3.  Business  Approach 
(Kashiwagi,  2010) 

The  outsourcing  owner  who  transfers  risk  and  control  to  the  expert  vendor  will  get 
the  high  performer.  It  is  the  only  win-win  situation.  The  high  performer  will  preplan,  manage 
and  minimize  risk  that  they  do  not  control  to  finish  on  time  and  maximize  their  profit,  do  the 
project  once,  and  get  paid  and  not  go  back  to  redo  or  fix  problems.  This  is  the  efficient  best 
value  system. 

Best  Value  Performance  Information  Procurement  System 
(PIPS) 

After  identifying  the  requirement  to  have  a  new  system/environment  that  reflected  the 
best  value  environment,  the  GSA  selected  the  best  value  PIPS  system  to  move  from  the 
price  based  to  the  best  value  environment  (GSA  contract  #:  GS06P09GYD0027).  The 
documented  performance  of  the  PIPS  system  included: 

16  years  of  testing  (1994-present)  delivering  700+  construction  services 
projects  valued  at  over  $800  million. 

Research  funding  of  $8.5  million. 

Minimized  client  risk/project  management  activities  by  up  to  90%. 

Maximized  vendor  profit  by  up  to  100%,  at  no  additional  cost  to  the  client. 

Delivered  performance  of  98%  on  time,  no  contractor  generated  cost  and 
time  deviations,  and  meeting  client’s  expectations. 

Arizona  State  University  (ASU)  moved  PIPS  into  non-construction  areas 
including  the  delivery  of  food  services,  IT  networking,  IT  data  centers,  help 
desks,  sports  marketing,  gym  equipment,  document  control,  long  distance 
education  services,  and  furniture  buys.  ASU  received  investments  of  $100 
million  over  ten  years  due  to  the  change  of  system  environment,  from  price 
based  to  best  value. 

The  Dutch  infrastructure  agency  used  PIPS  to  deliver  $1  billion  of  highway 
infrastructure  to  solve  their  problems  with  the  delivery  of  construction. 

The  Bank  of  Botswana  used  PIPS  to  deliver  a  critical  bank  facility  and  found  it 
tremendously  better  than  the  traditional  process. 

The  State  of  Alaska  is  delivering  a  $200  million  Electronic  Resource  Planning 
system  using  PIPS. 
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PIPS  is  a  licensed  structure/process  from  Arizona  State  University  developed  by 
Dean  Kashiwagi  and  the  Performance  Based  Studies  Research  Group  (PBSRG.)  PIPS 
has  three  main  phases  (Kashiwagi,  2010): 

1 .  Phase  I:  Selection  of  the  best  value  vendor. 

2.  Phase  II:  Pre-award/pre-planning  and  creation  of  the  risk  management  plan 
(RMP)  and  the  weekly  risk  report  (WRR.) 

3.  Phase  III:  Project  delivery  by  the  risk  management  of  deviation  of  time  and 
cost. 


Figure  4.  PIPS/PIRMS  Phases 

(Kashiwagi,  2010) 


During  the  early  development  of  PIPS,  the  selection  phase  was  identified  as  the 
most  important  phase.  As  more  tests  were  run,  it  was  identified  that  the  pre-award, 
preplanning  phase  was  a  more  critical  phase.  The  risk  management  capability  of  PIPS 
became  obvious,  and  the  term,  the  Performance  Information  Risk  Management  System 
(PI RMS)  was  created  to  allow  owners  to  use  the  risk  management  capability  of  the  system 
even  though  they  would  select  their  vendor  using  the  price  based  selection  mode. 

PIPS  has  six  major  filters  in  the  selection  phase  to  ensure  performance  (Figures  6 

and  7): 

1 .  Requires  the  vendors  to  prove  the  potential  to  perform  through  documented 
past  performance  including  the  performance  of  critical  personnel  (project 
manager),  and  critical  sub-vendors  (engineering,  professional  consultants,  or 
other  crafts). 

2.  Capability  to  do  the  project.  This  submittal  is  a  blind  review  of  the  vendor’s 
capability  to  do  the  project: 

a.  Requires  the  vendors  to  show  an  understanding  of  the  scope  of  the 
project  in  terms  of  the  biggest  technical  risks  in  a  concise  and  short 
explanation  (two  page  submittal).  The  high  performers  should  identify  the 
major  technical  challenges  of  the  project  and  what  makes  them  capable 
of  minimizing  the  risk  of  nonperformance. 


b.  Risk  AssessmentA/alue  Added  (RAVA)  submittal  that  forces  the  vendor  to 
identify  the  risk  that  the  vendor  does  not  control,  and  how  they  will 
manage  and  minimize  that  risk  so  it  does  not  occur.  It  also  asks  the 
vendors  to  document  dominant  added  value  being  offered  by  the  vendor 
(that  is  not  in  the  client’s  specified  scope,  and  will  create  a  dominant 
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difference  in  project  value)  that  makes  them  different  from  their 
competitors. 

c.  The  vendor  is  also  asked  for  a  milestone  schedule,  and  how  they  will 
measure  their  performance  during  the  project. 

d.  The  vendor  will  also  submit  a  cost  breakout. 

3.  The  interview  of  the  critical  personnel  of  the  vendor  to  identify  the  person’s 
relative  vision,  ability  to  predict  things  before  they  occur,  preplan  and  their 
capability  to  be  accountable. 

4.  A  cost  check  to  ensure  the  best  value  is  not  overly  expensive. 

5.  Prioritization  of  the  best  value  based  on  the  capability  to  perform  (the  past 
performance  information,  scope  rating,  RAVA  rating,  milestone  schedule 
rating,  rating  on  performance  measurement  system,  interview  rating  and 
price). 

6.  Pre-award  Phase  where  just  the  best  value  vendor  creates  a  risk 
management  plan  (RMP)  and  a  weekly  risk  report  (WRR)  that  they  will  use  to 
manage  and  minimize  the  deviation  of  the  project.  The  WRR  and  RMP  track 
risks  that  the  vendor  does  not  control.  This  is  the  key  mechanism  in 
PIPS/PIRMS,  and  is  the  regulator  that  ensures  that  risk  and  control  is 
transferred  to  the  vendor.  This  becomes  a  key  component  of  the  contract, 
and  decommissions  any  attempts  for  the  owner’s  PM  to  manage,  direct,  and 
control  the  vendor.  This  results  in  an  alignment  of  resources,  as  it  is  in  the 
best  interest  of  all  parties. 

A  major  paradigm  shift  is  the  movement  from  management,  direction,  and  control  to 
quality  control/quality  assurance.  This  movement  assumes  that  the  vendor  has  no  technical 
risk,  and  therefore  will  concentrate  on  managing  and  minimizing  the  risk  they  don’t  control. 
This  will  be  more  fully  explained  in  a  following  section. 

The  vendor  then  writes  their  own  contract  (technical  requirement,  legal  requirements 
of  the  owner,  risk  management  plan  and  weekly  risk  control  report.)  The  project  is  then 
awarded.  The  vendor  self  manages  themselves  based  on  the  contract,  managing  and 
minimizing  the  cost  and  time  deviation  of  the  project.  At  the  end  of  the  project,  the  vendor  is 
rated.  The  rating  becomes  50%  of  the  vendor’s  future  rating. 
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Figure  5.  PIPS/PIRMS  Filters 
(Kashiwagi,  2010) 


Action? 

-  Minimize  da u  flow 

•  Minimize  analysis 

•  Minimize  control 


Past  Performance 
information  (PPI) 


identify  VJui* 

R  <-  Mmim7’eUi<*< 
M)=  S^ir^aSUI-HTltHflt 


Requirements 
<065.  06.  CMAft,  OBO> 


i 


Capability  To  Do  Project 
Interview  Key  Personnel 
Matrix  Comparison 

Best  Value  for  the  Lowest 
Cost  (Determined  by 
Vendors) 

)  Preplanning.  Risk 
Management 


Efficient  Service  Managed  and 
Controlled  by  Vendor 

JL  2 


J 


Figure  6.  PIPS/PIRMS  Self  Regulating  Closed  Loop 

(Kashiwagi,  2010) 


A  Paradigm  Shift:  Understanding  that  Expert  Vendors  Have  no 
Technical  Risk 

A  major  departure  from  the  traditional  project  management  practices  is  the 
understanding  and  handling  of  risk.  Information  Measurement  Theory  (IMT)  identifies  that 
by  definition,  high  performance/expert  personnel  have  minimal  or  no  technical  risk 
(Kashiwagi,  2010).  If  there  is  technical  risk,  it  is  only  because  the  client  hired  a  vendor  who 
does  not  have  the  expertise  and  therefore  is  not  capable  of  minimizing  risk.  Instead  of 
managing,  directing,  and  controlling  the  vendor,  the  owner  is  now  creating  a  new 
environment,  where  the  vendor  is  identified  as  the  expert.  Therefore,  the  new  environment 
minimizes  all  management,  direction,  and  control  of  the  vendor.  This  simple  but  difficult 
change  in  paradigm,  forces  the  transfer  of  risk  and  control  to  the  expert,  and  aligns  all 
resources. 

The  authors  propose  that  the  impact  of  unforeseen  conditions  can  be  minimized  if 
experts  are  used  to  manage  and  minimize  the  risk  that  they  don’t  control.  The  only  risk  high 
performers  have  is  risk  that  they  do  not  control  (risk  that  is  brought  by  other  participants, 
mainly  the  client  in  the  form  of  over-expectations,  items  outside  of  the  scope,  decision 
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making  by  other  participants  at  the  wrong  time  during  the  process,  and  the  changing  of 
expectations)  (Figure  8). 

The  new  paradigm  motivates  vendors  to  preplan  the  project  from  beginning  to  end  and 
know  the  risk  that  they  do  not  control  and  how  they  will  manage  that  risk  before  they  accept 
the  project.  By  deductive  logic,  a  system  that  increases  client  management,  direction,  and 
control  moves  the  activity  to  the  more  inexperienced  vendors  and  personnel  (Figure  8).  This 
results  in  lower  performance,  reactive  behavior,  minimum  standards  or  expectations,  and 
minimum  accountability. 

Price  based  contracts  emphasize  the  technical  risk  that  the  vendors  must  control. 
Price  based  contracts  attract  the  less  experienced,  and  makes  the  very  experienced  less 
competitive  (Figures  4  and  8).  Best  value  contracts  must  identify  and  communicate  the 
expectations  of  the  client  but  emphasize  the  requirement  of  the  vendors  to  manage  and 
minimize  the  risk  that  they  do  not  control,  thus  thinking  in  the  best  interest  of  the  client  and 
creating  a  “win-win.”  Price  based  contracts  must  cater  to  the  inexperienced  and  increase 
the  flow  of  information,  contract  documents,  and  client  management,  direction,  and  control. 
Best  value  contracts  cater  to  the  high  performing  contractors  who  need  minimal  information, 
who  act  in  the  best  interest  of  the  client  by  giving  high  technical  service  (no  technical  risk) 
and  manage  and  minimize  the  risk  that  the  vendor  does  not  control  through  the  use  of 
quality  control  and  risk  management  plans. 
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Figure  7.  Inexperienced  versus  Experienced  Vendor  Risk  Model 

(Kashiwagi,  2010) 

High  performers  and  experts  by  definition  are  rarely  surprised,  are  rarely  affected  by 
“unforeseen  conditions,”  and  if  there  are  “unforeseen  conditions,”  high  performers  have  a 
preplanned  solution  that  they  can  easily  identify,  manage,  and  minimize  the  cost  and  time 
deviation.  Due  to  the  low  performing  mentality  of  the  construction  industry,  and  resulting  low 
performance  of  the  industry  participants,  the  risk  of  “unforeseen  conditions”  has  increased, 
and  instead  of  requiring  high  performance  and  the  minimization  of  risk  that  the  vendor  does 
not  control,  the  industry  has  created  a  transaction  based  solution  of  client  management, 
direction  and  control,  which  increases  the  number  of  participants  who  do  not  have  the 
capability  to  minimize  the  risk.  These  participants  create  silos  where  they  verify  and 
manage  the  vendors  schedule  and  cost,  negotiate  prices,  and  approve  all  deviations  in 
schedule  or  materials.  In  this  inefficient  system,  the  inexperienced  contractor  becomes 
more  competitive  than  the  high-performance  contractor  with  very  experienced  personnel, 
who  cannot  efficiently  do  their  project  in  an  environment  of  redundant  and  un-needed 
transactions. 

These  deductive  concepts  have  not  only  been  confirmed  through  the  15  years  of 
testing,  but  have  been  identified  in  other  non-construction  industries.  By  minimizing  the 
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management,  direction,  and  control  of  subject  matter  experts  (SME),  transferring  the  risk 
and  control  to  the  vendor,  by  forcing  the  vendor  to  write  the  contract  which  requires  an 
expert  vendor,  and  by  forcing  the  vendor  to  manage  and  minimize  the  risk  they  do  not 
control,  a  structure  has  been  created  which  aligns  resources  and  minimizes  the 
transactions.  The  regulator  of  the  structural  change  is  the  weekly  risk  report  (WRR)  and  the 
risk  management  plan  (RMP.)  They  create  transparency,  which  minimizes  the  need  for 
management,  direction  and  control. 

Method  of  Measurement 

The  method  of  measurement  of  performance  utilized  by  PIPS  is  a  novel  approach 
which  has  had  successful  results  in  the  past  five  years  (Kashiwagi,  2009).  The  assumption 
is  that  a  large  organization  delivering  services  with  project  managers  who  are  accustomed 
to  manage,  direct,  and  control,  will  have  a  difficult  time  consistently  and  doing  timely 
reporting  and  analysis  of  performance  due  to  the  inefficiency  of  a  management  based 
system.  PIPS  identifies  who  should  be  at  risk  (expert),  and  forces  the  measurement  of 
performance  (deviation  of  project  time  and  cost)  by  the  vendors.  The  deviations  are  then 
reviewed  for  accuracy  by  the  client/buyer’s  professional  (quality  assurance.)  The 
information  goes  directly  to  the  top  decision  maker  in  the  organization,  bypassing  the  normal 
filtering  system  in  a  bureaucracy.  A  simplistic  use  of  spreadsheets  does  the  following: 

1 .  Identifies  the  deviation  rates  of  all  projects. 

2.  Identifies  the  top  ten  riskiest  projects,  and  which  participant  in  the  supply 
chain  caused  the  deviation. 

3.  Measures  all  participants  in  the  supply  chain,  giving  a  relative  performance 
rating  based  on  deviations  and  performance. 

The  PIPS  measuring  system  overcomes  the  major  obstacles  large  organizations 

have: 

1 .  Subjective  filtering  of  measurements. 

2.  Lack  of  timely  reporting. 

3.  Lack  of  time  to  do  accurate  reporting. 

GSA  Strategic  and  Tactical  Plans  to  Implement  New 
Environment 

The  GSA  signed  a  five-year  contract  with  the  Performance  Based  Studies  Research 
Group  (PBSRG)  to  implement  the  new  best  value  system.  Previous  research  results 
(Kashiwagi,  2010)  identified  that  both  a  strategic  plan  and  a  tactical  plan  are  required  to 
successfully  make  the  transformation.  Previous  results  identified  the  following  priorities: 

Identification  of  a  core  group  of  visionaries  who  could  understand  the 
theoretical  change  in  systems,  and  be  systems  managers  and  managers  of 
personnel. 

Education,  development  of  the  core  group  of  visionaries  to  learn  how  to  use 
the  system. 

Implementing  and  modifying  PIPS  to  fit  the  environment  of  the  owner. 
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4.  General  education  to  the  owner’s  organization  of  the  PIPS  system. 

The  strategic  plan  encompasses  the  first  two  objectives,  and  the  tactical  plan  is  the 
second  two  objectives.  The  tactical  plan  (second  two  objectives)  cannot  be  implemented 
without  the  strategic  plan.  The  strategic  plan  therefore,  must  include  the  following: 

1 .  The  development  of  the  visionaries. 

2.  Theoretical  education  of  the  visionaries. 

3.  Job  transformation  of  the  visionaries  from  project  managers  to  educators  and 
systems  managers. 

4.  Changing  of  the  effectiveness  of  the  chain  of  command. 

5.  The  visionary  group  must  learn  to  apply  the  concept  of  PIPS  to  their 
organization’s  transformation. 

6.  Documentation  of  the  transformation  by  the  system. 

7.  Peer  review  by  other  visionaries  located  at  other  organizations  who  are 
working  toward  the  same  goals. 

8.  Development  of  measurements  and  a  schedule  showing  the  improvement  in 
performance  and  value. 

The  tactical  plan  must  include: 

1 .  The  modification  of  the  PIPS  to  move  the  owner’s  organization  without 
increasing  resistance  due  to  the  change  of  efficiency  and  structure. 

2.  Education  on  PIPS  to  both  vendors  and  PMs  running  the  system. 

3.  Prototype  testing  and  implementing  PIPS  by  core  team  visionaries. 

4.  Design  of  the  information  system. 

If  the  concept  of  transferring  risk  and  control  to  the  expert  vendor,  and  aligning  the 
resources  in  the  entire  supply  chain  through  measurement  of  deviation  is  accurate,  this 
system  transformation  is  not  industry  specific.  It  is  a  system  regulated  by  measurement, 
which  aligns  all  the  participants  in  the  supply  chain,  minimizing  the  transactions  and  forcing 
experts  to  be  accountable.  It  will  bring  discomfort  if  a  subject  matter  expert  (SME)  is 
misaligned  or  does  not  currently  have  accountability.  Therefore  the  transformation  has  to  be 
“gentle”  and  evolutionary. 

Progress  of  the  GSA 

The  implementation  of  PIPS  in  the  GSA  has  been  relatively  optimized  due  to  the 
following: 

The  head  of  the  core  team  visionaries,  the  region  director  of  the  organization, 
was  already  attempting  to  transform  the  organization  to  a  measured 
organization.  His  strategic  goals  of  efficiency  and  effectiveness  of  both 
vendors  and  the  GSA  organization  was  already  in  place. 

The  PMs  of  the  core  team  were  identified  and  selected  based  on  the 
Information  Measurement  Theory  (IMT)  and  therefore  were  attempting  to  use 
the  concepts  of  PIPS  before  the  transformation  effort. 
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This  is  the  first  time  in  16  years  of  testing  that  both  a  PM  and  procurement 
officer  were  original  members  of  the  visionary  core  team,  and  the  director 
was  already  attempting  the  transformation. 

4.  This  is  the  first  time  that  the  PIPS  was  selected  through  application  of  PIPS, 
thus  confirming  to  the  core  team  that  PIPS  was  dominant  in  its  ability  to 
transform  organizational  environment/systems. 

First  Measurement  of  Existing  Performance 

The  core  team  selected  8  projects  where  information  was  readily  available  to  identify 
the  performance  of  the  existing  environment.  The  performance  measurements  include: 

1 .  Average  cost/scope  of  projects:  $526,992 

2.  Average  duration  of  projects:  152.5  days 

3.  Cost  deviation  of  projects  (percentage):  7% 

4.  Time  deviation  of  projects  (percentage):  231  % 

5.  Customer  satisfaction  (1  -10  rating,  10  being  optimal):  6.5 

The  core  team  is  also  interested  in  the  following  measurements  that  will  be  collected 
through  surveys: 

1 .  Vendor  profit  margin:  TBD 

2.  Vendor  rating  of  delivery  system:  TBD 

3.  Vendor  perception  of  new  system  (1-10  rating):  TBD 

4.  Number  of  projects  a  PM  is  responsible  for:  TBD 

The  GSA’s  next  step  is  to  complete  six  (6)  test  projects  and  collect  data  to  confirm 
increased  performance  with  the  new  best  value  PIPS  system.  The  Contractor  and  PM  shall 
rate  the  following  before  and  after  on  the  traditional  system  vs.  the  new  best  value  PIPS 
system: 

1.  Effectiveness  (deviations.) 

2.  Value  of  preplanning  by  vendor  as  perceived  by  both  the  vendors  and  the 
GSA  PM. 

3.  Value  of  vendor  managing  and  minimizing  the  risk  that  the  vendor  does  not 
control  as  perceived  by  all  participants. 

4.  Vendor’s  profit  margin  maximization. 

5.  Accountability  of  all  the  participants  as  perceived  by  all  participants. 

6.  Successfulness  and  impact  of  the  transfer  of  risk  and  accountability  to  the 
vendor. 

7.  Project  coordination  by  the  vendor  with  the  client. 

8.  Minimization  of  surprises. 
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Schedule  of  Implementation 

It  is  a  five-year  tactical  plan: 

1 .  Year  1 :  set  up  core  team  structure.  Run  the  first  tests  with  core  team  and  a 
few  PMs. 

2.  Year  2:  set  up  the  Directors  Report  and  expand  both  the  running  of  the  entire 
process  and  the  risk  management  reporting  (which  measures  the 
performance  of  projects). 

3.  Years  3-5:  expand  implementation  within  organization.  Visionary  core  team 
becomes  a  subject  matter  expert  (SME)  to  assist  in  the  transformation  of 
other  organizations. 

Analysis  of  the  Effort 

This  research  effort  is  using  the  deductive  approach  (confirmatory)  instead  of  the 
inductive  approach  (exploratory.)  The  success  of  the  project  will  be  determined  by 
measurements  of  observation  which  minimize  subjectivity  as  much  as  possible.  The 
following  are  observations  of  the  effort  thus  far: 

1 .  PIPS  has  been  identified  by  a  GSA  selection  process  as  the  only  option  with 
documentation  of  proven  success  to  transform  an  organization’s  environment 
from  a  management,  direction,  and  control  environment  to  a  best  value, 
alignment,  leadership  based  environment. 

2.  A  large  federal  organization  who  is  constrained  by  federal  law,  will  implement 
the  PIPS  process  for  selection  of  vendor,  and  contract  administration. 

3.  A  visionary  core  team  has  been  organized  that  is  optimal  in  terms  of  a  high- 
ranking  visionary  leader,  and  visionary  PM  and  procurement  components. 

4.  For  the  first  time,  strategic  and  tactical  plans  have  been  drawn  up  and  will  be 
used  in  the  research  test. 

Conclusion 

The  GSA  Heartland  region  is  implementing  an  advanced  and  theoretically  sound 
best  value  delivery  process  to  transform  the  system  from  a  price  based  to  a  best  value 
environment.  The  major  objectives  include:  minimization  of  management,  direction,  and 
control  transactions,  the  transfer  of  risk  and  control  to  vendors  who  can  minimize  the  risk, 
measurement  of  performance  of  the  vendors  and  the  GSA  organization,  and  to  measure  an 
increase  in  performance  and  value  of  the  services  being  delivered.  A  core  group  of 
visionaries  are  attempting  to  transform  the  organizational  approach  from  one  of 
management  of  personnel  to  a  systems  management,  where  performance  measurements 
drive  alignment  of  resources.  This  is  a  significant  effort  for  a  large  federal  organization  that 
normally  is  management  based  and  has  difficulty  in  minimizing  bureaucracy. 
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2003  -2010  Sponsored  Research  Topics 


Acquisition  Management 


Acquiring  Combat  Capability  via  Public-Private  Partnerships  (PPPs) 

BCA:  Contractor  vs.  Organic  Growth 
Defense  Industry  Consolidation 
EU-US  Defense  Industrial  Relationships 

Knowledge  Value  Added  (KVA)  +  Real  Options  (RO)  Applied  to  Shipyard 
Planning  Processes 

Managing  the  Services  Supply  Chain 

MOSA  Contracting  Implications 

Portfolio  Optimization  via  KVA  +  RO 

Private  Military  Sector 

Software  Requirements  for  OA 

Spiral  Development 

Strategy  for  Defense  Acquisition  Research 

The  Software,  Hardware  Asset  Reuse  Enterprise  (SHARE)  repository 


Contract  Management 


Commodity  Sourcing  Strategies 
Contracting  Government  Procurement  Functions 
Contractors  in  21st-century  Combat  Zone 
Joint  Contingency  Contracting 

Model  for  Optimizing  Contingency  Contracting,  Planning  and  Execution 

Navy  Contract  Writing  Guide 

Past  Performance  in  Source  Selection 

Strategic  Contingency  Contracting 

Transforming  DoD  Contract  Closeout 

USAF  Energy  Savings  Performance  Contracts 

USAF  IT  Commodity  Council 

USMC  Contingency  Contracting 


Financial 


Management 


Acquisitions  via  Leasing:  MPS  case 


Budget  Scoring 


Budgeting  for  Capabilities-based  Planning 
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Capital  Budgeting  for  the  DoD 

Energy  Saving  Contracts/DoD  Mobile  Assets 

Financing  DoD  Budget  via  PPPs 

Lessons  from  Private  Sector  Capital  Budgeting  for  DoD  Acquisition  Budgeting 
Reform 

PPPs  and  Government  Financing 
ROI  of  Information  Warfare  Systems 
Special  Termination  Liability  in  MDAPs 
Strategic  Sourcing 

Transaction  Cost  Economics  (TCE)  to  Improve  Cost  Estimates 


Human  Resources 


Indefinite  Reenlistment 

Individual  Augmentation 

Learning  Management  Systems 

Moral  Conduct  Waivers  and  First-tern  Attrition 

Retention 

The  Navy’s  Selective  Reenlistment  Bonus  (SRB)  Management  System 
Tuition  Assistance 


Logistics  Management 


Analysis  of  LAV  Depot  Maintenance 
Army  LOG  MOD 
ASDS  Product  Support  Analysis 
Cold-chain  Logistics 

Contractors  Supporting  Military  Operations 
Diffusion/Variability  on  Vendor  Performance  Evaluation 
Evolutionary  Acquisition 

Lean  Six  Sigma  to  Reduce  Costs  and  Improve  Readiness 
Naval  Aviation  Maintenance  and  Process  Improvement  (2) 

Optimizing  CIWS  Lifecycle  Support  (LCS) 

Outsourcing  the  Pearl  Harbor  MK-48  Intermediate  Maintenance  Activity 
Pallet  Management  System 
PBL  (4) 

Privatization-NOSL/NAWCI 
RFID  (6) 
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■  Risk  Analysis  for  Performance-based  Logistics 

■  R-TOC  AEGIS  Microwave  Power  Tubes 

■  Sense-and-Respond  Logistics  Network 

■  Strategic  Sourcing 

Program  Management 

■  Building  Collaborative  Capacity 

■  Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module  Acquisition 

■  Collaborative  IT  Tools  Leveraging  Competence 

■  Contractor  vs.  Organic  Support 

■  Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 

■  KVA  Applied  to  AEGIS  and  SSDS 

■  Managing  the  Service  Supply  Chain 

■  Measuring  Uncertainty  in  Earned  Value 

■  Organizational  Modeling  and  Simulation 

■  Public-Private  Partnership 

■  Terminating  Your  Own  Program 

■  Utilizing  Collaborative  and  Three-dimensional  Imaging  Technology 

A  complete  listing  and  electronic  copies  of  published  research  are  available  on  our  website: 
www.acquisitionresearch.org 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


629 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


630 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 

555  DYER  ROAD,  INGERSOLL  HALL 

MONTEREY,  CALIFORNIA  93943 


www.acquisitionresearch.org 


VHA*Vm 


PM  Model  of  the  Future 
Source  Selection,  Risk  Management, 
Contract  Administration 


Jacob  Kashiwagi 

Program  Manager,  Performance  Base  Studies  Research  Group 

Arizona  State  University 


PBSRG 

(Performance  Based  Studies  Research  Group) 


•  700  Procurements 

•  $808  Million  Construction  services 

•  $1.7  Billion  Non-construction  services 

•  $1 .3B  Euro  ($2B)  construction  test  ongoing  in  the 
Netherlands 

•  Africa/Southeast  Asia/Australia 

•  ASU  procurement  -  $100M  cash  savings  over  10 
years 


GSA  Heartland  Region  implementation  in  2009 

98%  Customer  satisfaction,  90%  of  PM/RM 
transactions  minimized 


Increased  vendor  profits  and  decreased  cost 


Atonstf  action  QunErs. 
■Issociart&n  of  Am  erica 
2007  Gold  Award 
Design  and  Construction 


Acquisition  Research  Program;  Creating  Synergy  for  Informed  Change 


Naviil  iii]  unic  Sl  Eiti.nl 

MhnSfrej.  <1A 


Industry  Structure 


High 


CD 

o 

c 

CD 


£ 

&_ 


0 

Q_ 


III.  Neqotiated-Bid 

Owner  selects  vendor 
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Major  Points 


•  Inefficient  to  manage  and  direct  the  expert 


•  An  expert  should  be  able  to  make  things  simple  and 
easy  to  understand 


•  Most  risk  on  projects/services  comes  from  the 
misperception  of  the  client 


•  Vendors  are  working  off  of  a  demand  and  supply 
model 
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Best  Value  System 

Performance  Information  Procurement  System 
(PIPS) 

PM  model,  Risk  Management  model 


PHASE  1: 

PHASE  2: 

PHASE  3: 

SELECTION 

PRE-PLANNING 

QUALITY 

CONTROL 

MANAGEMENT 

BY  RISK 
MINIMIZATION 

Buyer  Controls  Vendor  Through  Contract 


Vendor  Manages/Minimizes  Risk  With  Contract 
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Principles  of  Logic  and  Common 
Sense 

•  Efficiency  (less  work  and  better  results) 

•  Performance  measurement  (system  generated 
measurements) 


•  Transfer  of  risk  and  accountability  (to  vendors  who  preplan 
and  have  expertise) 


•  Minimize  transactions  (up  to  90%  of  our  effort,  allowing  us  to 
measure  and  analyze) 
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US  Army  MEDCOM  Bureaucracy 


Simplification 


Regional 

Director 

Regiona 

1  Director 

Hospital 


Contractor  1 


Contractor  2 


Contractor  3 


Contractor  4 


Contractor  5 


Contractor  6 


Contractor  7 


Contractor  8 


Contractor  9 


Contractor  10 


Contractor  1 1 


Contractor  12 


Contractor  13 


Contractor  14 


Contractor  15 


Contractor  16 
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MEDCOM  ON-GOING  OVERVIEW 


DIVISION  OVERVIEW 

1/15/2010 

Original  projects  budget 

$711,572,485.02 

Current  estimated  cost 

$740,069,776.77 

Estimated  cost  over  budget 

$28,497,291.75 

%  estimated  cost  over  budget 

4.00% 

PROJECT  OVERVIEW 

Total  number  of  projects 

265 

%  projects  on  time 

57% 

#  of  jobs  delayed 

114 

%  projects  on  budget 

66% 

#  of  jobs  over  awarded  budget 

90 

AVERAGE  PROJECT 

%  over  Awarded  Budget 

4.00% 

%  over  budget  due  to  owner 

2.86% 

%  over  budget  due  to  contractor 

0.00% 

%  over  budget  due  to  unforeseen 

1.14% 

%  Delayed 

18.27% 

%  Delayed  due  to  owner 

12.83% 

%  Delayed  due  to  contractor 

1.22% 

%  Delayed  due  to  unforeseen 

_ 4,22% _ 
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TOP  TEN  LIST 


TOP  10  RISK  RANKING  PROJECTS  1/15/2010 

No. 

Project 

Location 

Risk  # 

Contractor 

#  Weeks  on  Top 
10 

NTP  of  Project 

1 

Addition  to  Third  Floor  Woman's  Health 
Care  Suite 

Ft.  Lewis,  WA  (MAMC) 

42.63 

CON  A 

56 

10/18/2007 

2 

Bathroom  Conversions,  Bldg  9200 

Ft  Benning,  GA 

35.10 

CON  A 

14 

9/19/2008 

3 

WP/Rpl  Surgical  Supply  Cartlifts 

Ft.  Stewart,  GA  (WACH) 

14.23 

CON  A 

8 

9/30/2008 

4 

Renew  Health  Clinic,  Building  990 

Yuma  Proving  Grounds,  AZ 

13.53 

CON  A 

9 

9/11/2007 

5 

Misc.  Medical  Repair  Projects  '08 

Ft.  Bragg,  NC  (WAMC) 

13.10 

CONI 

2 

9/29/2008 

6 

Rep,Reconf  for  TBI  Program 

Schofield  Barracks,  HI 

12.73 

CON  A 

40 

10/7/2008 

7 

Convert  Constant  Volume  to  VAV 

Forest  Glenn,  MD  (WRAIR) 

11.46 

COND 

6 

10/29/2006 

8 

Replace  Elevators 

Ft.  Belvoir,  VA  (DACH) 

10.46 

CONE 

8 

5/2/2008 

9 

Repair  Budge  Dental  Clinic 

Ft.  Sam  Houston,  TX 
(BAMC) 

10.34 

COND 

2 

9/30/2007 

10 

Expand  Patient  Administration  Division 

Ft.  Lewis,  WA  (MAMC) 

9.12 

CON  A 

5 

10/15/2009 
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Modifications  and  Risks 


AlVAEd 

DS  <&  MODIFICATIONS 

No. 

Award  Modification 

I>a  te 

Type 

Days 

n 

Description 

1 

Award  1 

1/2/2007 

Award 

2 

|  Modification  1  — 

01/05/07 

' 

250,000.00 

)  Risk  1 

3 

Modification  2 

3/8/Z007 

(20) 

sTtmuuFCg 

Risk  2 

4 

Modification  3 

5/10/2007 

V 

$  150,000.00 

Risk  3 

Total  Contract: 

$  10,000,000.00 

9 1  Billed: 

50% 

0  0  Completed 

60% 

No 

- 1 

Date 

Entered 

- 

Risk  Items 

i 1 

Plan  to  Minimize  Risk 

- 1 

Planned 

Resolution 

Date 

- 1 

Actual  Date 

Resolved 

i - 1 

Impact  Days  to 
Critical  Path 
(Calendar) 

- i 

Impact  to 
Cost 

; ?  ’ : -  - :  -  ; .  1  -  -  ■:  '  ■  : 

:RSjte  APliapi:  1)  Pi  obJcm  track  0  found -tfihy  is  tlii  sail 

i 

•;  0  : 

3/17 mos 

EVIA.MDI  C 1  •  D1  jp|  f  JV  1  • '  - 1  - 1  •  1  •  *  • r  • 1  “ 1  - *  -  *  • 

:  uiie*j)«ic(ed  risk?- 2)  Vfli&t  will  bcdoricitbrifiiiiimfre  : : 

$ 

:;:10.00Q 

C AMWErL  C :  “3  9  R:  ft  : 

;tiiis?3)  VVItois  responsible fa i :th e pla n ? ■  41  V/hat  of  :  : 

:  o-r:l/  • 

’  r  '  '  :  ;  : 

:  U  •  : 

1 

12/25/2007 

Delavin  V/orkplan 

.[  .WH  l:1h  ISL  : 

Risk  Plan  B 

1/3/2007 

1/3/2007 

Av  < 

s 

250,000 

2 

3/1/2007 

Contaminated  Material 

Risk  Plan  C 

3/B/2007 

3/7/2007 

© 

1  WJ  I1—11—  U 

3 

4/30/2007 

Scope  Change 

Risk  Plan  D 

5/5/2007 

5/6/2007 

Sr 

$ 

150,000 
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On-Going  Projects:  Regional  Performance  Lines 


REGION  OVERVIEW 

CHPPM 

PRMC 

AMEDD 

SRMC 

WRMC 

MRMC 

NRMC 

AFIP 

Average 

Total  Number  of  Projects 

3 

28 

7 

49 

38 

26 

27 

1 

22 

Total  Awarded  Budget 

$27,782,738 

$  44,409,340 

$  18,452,757 

$148,750,286 

$133,683,925 

$  60,138,879 

$118,356,664 

$9,754,941 

$70,166,191 

Current  Cost 

$27,910,447 

$  47,054,360 

$20,198,239 

$155,289,910 

$139,654,057 

$  63,259,537 

$121,621,485 

$9,823,830 

$73,101,483 

PROJECT 

INFORMATION 

CHPPM 

PRMC 

AMEDD 

SRMC 

WRMC 

MRMC 

NRMC 

AFIP 

Average 

%  Projects  On  Time 

100% 

64% 

57% 

53% 

34% 

31% 

22% 

0% 

45% 

%  Projects  On  Budget 

67% 

61% 

86% 

47% 

53% 

46% 

33% 

0% 

49% 

%  Delayed 

0.00% 

22.50% 

13.70% 

15.80% 

21.50% 

37.90% 

32.80% 

0.06% 

18% 

%  Over  Budget 

0.46% 

5.96% 

9.46% 

4.40% 

4.47% 

5.19% 

2.76% 

0.71% 

4% 

Average  Risk  Number 

1.01 

2.14 

1.52 

1.92 

4.33 

2.77 

3.05 

1.07 

2.2 

GENERAL 

INFORMATION 

CHPPM 

PRMC 

AMEDD 

SRMC 

WRMC 

MRMC 

NRMC 

AFIP 

Average 

#  of  QA’s 

1 

14 

5 

17 

14 

8 

14 

1 

9 

#  of  Projects  per  QA 

3.0 

2.0 

1.4 

2.9 

2.7 

3.3 

1.9 

1.0 

2 

#  of  Facilities 

1 

2 

4 

9 

8 

9 

10 

1 

6 

Accurate  Weekly  Risk 
Reports 

66% 

60% 

50% 

60% 

45% 

37% 

50% 

0% 

46% 

Risk  Management  Plans 

100% 

68% 

66% 

77% 

60% 

55% 

59% 

100% 

73% 

Average  Risk  Resolving 
Time  (days) 

0.77 

12.2 

25.3 

19.4 

23.3 

19 

22 

1.5 

15 

Projects  with  risk  #  more 
than  7 

0 

1 

0 

3 

9 

2 

2 

0 

2 
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CONTRACTOR  PERFORMANCE  LINES 


CONTRACTOR  OVERVIEW 

CON  A 

CON  B 

CON  C 

COND 

CONE 

CONH 

Total  Awarded  Budget 

$  311,698,895 

$  68,513,436 

$  52,432,079 

$  208,010,504 

$  80,506,795 

$  1,200,377 

Current  Cost 

$  322,159,798 

$  71,286,239 

$  54,026,603 

$  215,332,223 

$  86,837,793 

$  1,315,698 

Over  Budget 

$  10,460,903 

$  2,772,803 

$  1,594,523 

$  7,321,719 

$  6,330,998 

$  115,321 

PROJECT  OVERVIEW 

Total  Number  of  Projects 

146 

15 

7 

58 

35 

2 

%  Projects  On  Time 

66% 

40% 

29% 

55% 

29% 

50% 

#  of  Jobs  Delayed 

49 

9 

5 

26 

25 

1 

%  Projects  On  Budget 

75% 

60% 

14% 

59% 

51% 

50% 

#  of  Jobs  Over  Awarded  Budget 

37 

6 

6 

24 

17 

1 

AVERAGE  PROJECT 

#  of  Risks  per  Job 

1.01 

3.60 

3.71 

1.31 

2.23 

1.00 

Number  of  overdue  risks 

1 

0 

2 

1 

1 

0 

%  Over  Awarded  Budget 

3.36% 

4.05% 

3.04% 

3.52% 

7.86% 

9.61% 

%  over  budget  due  to  Contractor 

0.04% 

0.53% 

0.01% 

0.00% 

-0.42% 

0.00% 

%  over  budget  due  to  Unforeseen 

0.87% 

0.54% 

1.98% 

0.19% 

1.18% 

0.00% 

#  of  Days  Delayed 

60 

135 

351 

72 

145 

105 

#  days  delayed  due  to  Contractor 

4 

41 

4 

0 

16 

31 

#  days  delayed  due  to  Unforeseen 

10 

55 

61 

4 

58 

0 

Risk  Number 

2.83 

2.37 

2.76 

2.35 

2.16 

1.74 

%  Projects  missing 

i% 

6% 

0% 

0% 

0% 

0% 

%  Projects  with  no  RMP 

32% 

43% 

0% 

0% 

0% 

50% 
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FACILITY  PERFORMANCE  LINES 


FACILITIES  OVERVIEW  1/15/2010 

FACILITY  A 

FACILITY  B 

FACILITY  C 

FACILITY  D 

Total  Awarded  Budget 

$  36,551,271 

$  6,980,380 

$  4,312,272 

$  9,824,793 

Current  Cost 

$  36,678,980 

$  6,980,380 

$  4,312,272 

$  10,543,804 

Over  Budget 

$  127,709 

$ 

$ 

$  719,011 

PROJECT  OVERVIEW 

Total  Number  of  Projects 

4 

3 

3 

4 

%  Projects  On  Time 

100% 

100% 

100% 

25% 

#  of  Jobs  Delayed 

0 

0 

0 

3 

%  Projects  On  Budget 

75% 

100% 

100% 

25% 

#  of  Jobs  Over  Awarded  Budget 

1 

0 

0 

3 

AVERAGE  PROJECT 

#  of  Risks  per  Job 

0.25 

0.00 

0.00 

2.75 

Owner  Generated  Risks 

0.25 

0.00 

0.00 

1.25 

Number  of  overdue  risks 

0 

0 

0 

1 

%  Over  Awarded  Budget 

0.35% 

0.00% 

0.00% 

7.32% 

%  over  budget  due  to  Owner 

0.35% 

0.00% 

0.00% 

5.27% 

%  over  budget  due  to  Contractor 

0.00% 

0.00% 

0.00% 

0.00% 

%  over  budget  due  to  Unforeseen 

0.00% 

0.00% 

0.00% 

2.05% 

#  of  Days  Delayed 

0 

0 

0 

110 

#  of  days  delayed  due  to  Owner 

0 

0 

0 

19 

#  days  delayed  due  to  Contractor 

0 

0 

0 

29 

#  days  delayed  due  to  Unforeseen 

0 

0 

0 

62 

Owner  Rating 

10.00 

10.00 

10.00 

9.13 

Risk  Number 

1.01 

1.00 

1.00 

2.48 
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Arizona  State  University  Food  Services  Contract 


Vendor 

No 

Summary  Criteria 

Out  of 

Incumbent 

B 

C 

1 

RAVA  Plan 

10 

5.91 

7.09 

6.31 

2 

Transition  Milestone  Schedule 

10 

5.17 

6.96 

6.33 

3 

Interview 

25 

15.77 

16.78 

13.53 

4 

Past  Performance  Information  -  Survey 

10 

9.80 

9.99 

9.82 

5 

Past  Performance  Information  -  #/Clients 

Raw  # 

5.67 

3.00 

4.42 

6 

Past  Performance  Information  -  Financial 

10 

7.02 

8.67 

6.90 

7 

Financial  Rating 

10 

4.00 

8.00 

8.00 

8 

Financial  Return  -  Commissions 

Raw  $ 

$  30,254,170 

$  60,137,588 

$  64,000,000 

9 

Capital  Investment  Plan 

Raw  $ 

$  14,750,000 

$  20,525,000 

$  12,340,000 

10 

Equipment  Replacement  Reserve 

Raw  $ 

$  7,213,342 

$  4,100,001 

$  8,171,811 

Finanical  Totals 

r$  52,217,512 

r$  84,762,589 

r$  84,511,811 

$32M  more  over  ten  years 


Vendor 

No 

Summary  Criteria 

Weight/Out  of 

Incumbent 

Best  Value 

c 

1 

RAVA  Plan 

28 

16.55 

19.85 

17.67 

2 

Transition  Milestone  Schedule 

2 

1.03 

1.39 

1.27 

3 

Interview 

25 

15.77 

16.78 

13.53 

4 

Past  Performance  Information  -  Survey 

9 

8.82 

8.99 

8.84 

5 

Past  Performance  Information  -  #/Clients 

1 

1.00 

0.53 

0.78 

6 

Past  Performance  Information  -  Financial 

15 

10.53 

13.01 

10.35 

7 

Financial  Rating 

5 

2.00 

4.00 

4.00 

8 

Financial  Return  -  Commissions 

7 

3.31 

6.58 

7.00 

9 

Capital  Investment  Plan 

6 

4.31 

6.00 

3.61 

10 

Equipment  Replacement  Reserve 

2 

1.77 

1.00 

2.00 

l^lNpv 

nm 

100 

65.09 

78.13 

69.04 

nr* 
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After  1  Year:  Monitoring/Evaluation 
based  on  measurements 


•  Increase  sale  of  food  by  14% 

•  Increased  cash  to  ASU  by  23% 

•  Minimized  management  cost  by  80% 

•  Increased  customer  satisfaction  by 
37% 

•  Increased  capital  investment  by 
100% 


FY  06-07 
Incumbent 

FY  07-08  New 
Vendor 

Difference 

%  Difference 

No 

Category 

1 

Total  Revenue  ($M) 

$  27.02 

$  30.83 

$  3.81 

14% 

2 

Total  Return  &  Commissions  ($M) 

$  2.17 

$  2.67 

$  0.50 

23% 

3 

Captial  Investment  Contract  ($M) 

$  14.75 

$  30.83 

$  18.08 

109% 

4 

Captial  Investment  2006  vs.  2007  ($M) 

$  0.26 

$  5.70 

$  5.44 

2092% 

5 

ASU  Administration  (#  of  People) 

7 

1.5 

-5.5 

-79% 

6 

Customer  (Student)  Satisfaction  (1-10) 

5.2 

7.1 

1.9 

37% 

7 

Myster  Shopper  Satisfaction 

N/A 

9.6 

— 

— 
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Final  ASU  IT  Networking  Contract 


ASU  Maintenance 
Annual  Cost 

Qwest  Maintenance 
Annual  Cost 

Total  Annual  Qwest 
Savings 

Total  Qwest  Annual 
Value  Added  and  Savings 

$13,981,934 

$12,500,000 

1,481,934 

2,756,934 

•  ASU  IT  Networking  previously 
performed  in-house 

•  ASU  IT  Network  Details 

-  76,000  Students  and  Faculty 

-  5  yr.  Contract 

-  4  Different  Campuses 


Value  Added 

Estimated  Annual 

Value 

Voicemail  Integrated  w/email 

575,000 

Experts  in  other  areas  of  Qwest  to  draw  upon 

$75,000 

Reduction  of  "2000  sqft  ASU  office  space  &  utilities 

544,000  ! 

Sky  song  state-of-the-art  Network  Operations  Center 

$100,000  I 

New  Contact  Center  Solution 

5400,000 

University  Benchmarking 

550,000 

Measurement  &  Reporting 

S50?QOQ 

Engineering  &  Design 

$150,000 

Speech  Enabled  Voice  Messaging 

$25,000 

Conferencing  &  Collaboration  Capabilities 

$9,000 

IP  Fax  Capabilities 

$12,000 

Unified  Co  m  m  u  n  i  c  at  i  o  n  s  F  A  an  a  ge  me  lit  T ool  set 

$S5,000 

Green  initiatives  (Kw  savings) 

$200  000 

Totnl  AiMitioii.il  Eslimiileil  AmMi.il  Value  to  ASU: 


1 1.275.000 


TViLVul  jJ  LJJJIi: 
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Dominant  Information 


•  Dominant  Performance  Indicators 

-  Overall  cost  of  network 

-  Top  of  the  line  networking 

-  Network  Sustainability/Accessibility 

-  Customer  Satisfaction 

•  Documentation  of  Deviations  to  financials 

I 


Dev. 

Cap,  Exp. 

Maint. 

FOE  Costs 

Total 

Year  1  Exp. 

$4,100,000 

$1,652,000 

$  6,818,000 

$  12,570,000 

Ex.  Risk  X 

$  100,000 

$  100,000 

$ 

$ 

$ 

100,000 

Ex.  Risk  X 

$  100,000 

$  100,000 

$  (25,000) 

$ 

$ 

75,000 

Ex.  Risk  X 

$  50,000 

$ 

$  50,000 

$ 

$ 

50,000 

Ex.  Risk  X 

$  25,000 

$  25,000 

$ 

$ 

$ 

25,000 

New  Year  1 

$  275,000 

$  4,325,000 

$1,677,000 

$  6,818,000 

$12,820,000 

Dominant  Measurements 

ASU 

Current 

Qwest 
Value  Add 

Overall  Cost  of  Network 

Annual  IT  Spend  Ratio  (new  vs 
maintenance) 

17/83 

48/52 

Top-of-the-line  Networking 

%  Converged 

7% 

100% 

%  Mobility 

2% 

100% 

%  Equipment  not  out-of-date 

58% 

95% 

Network  Sustainability/ Accessibility 

%  Equipment  not  needing  replacement 
(Not  at  end-of-maintenance) 

88% 

100% 

Customer  Satisfaction 

Speed/Quickness  Available  (Wired  / 
Wireless): 

%  1Gb  -  Wired  Connections 

59% 

98% 

%  of  300Mb  -  Wireless  Connections 

8% 

32% 
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GSA  Region  6:  Five  Year  Plan 


•  Year  1: 

-  Set-up  core  team  structure 

-  Design-Build  IDIQ  Contract 

-  Education  when  needed 

-  2  Test  projects 

•  Year  1  Revised: 

-  7  Test  projects 

-  1 6  projects  with  Weekly  Risk  Reports 


•  Start  Small...  Grow  Slow 
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Why  is  GSA  Heartland  Region  Effort  so 
Significant? 

•  GSA  is  known  as  the  biggest  buyer  of  non-military  services  in  the  U.S. 

•  First  time  the  application  is  being  run  with  procurement  and  project 
management  champions 


•  Best  combination  of  PM/Procurement  action  officers  in  16  years  of 
testing 


•  Organization  who  has  already  been  developing  measurements  of 
performance  and  who  is  attempting  to  utilize  the  measurements 


•  Visionary  leadership  with  visionary  action  officers 


•  Biggest  potential  impact  on  industry  practices  of  any  of  testing 
organizations 
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GSA  Weekly  Risk  Report  Data 


No 

Criteria 

Unit 

1 

Total  Number  of  Projects 

# 

2 

Total  Awarded  Cost 

$ 

3 

Average  Awarded  Cost 

$ 

4 

Average  Procurement  Time 

Days 

PIPS 

Traditional 

4 

8 

$  1 ,970,792 

$  9,396,738 

$  492,698 

$  1,879,348 

65 

98 

Contractor  Reaction  to  PIPS 


•  Two  contractors  are  educating  their  people 

•  Changing  their  procedures  to  be  proactive 
and  identify  value  for  the  client 


•  Educating  others  on  the  benefits  of  being 
visionary 


Contractor  Perceptions  on  Best  Value 


No 

Criteria 

Unit 

1 

Effectiveness  of  the  vendor. 

(1-10) 

2 

Value  of  preplanning  by  the  vendor. 

(1-10) 

3 

Value  of  managing  &  minimizing  the  risk  that  the  vendor  does  not  control. 

(1-10) 

4 

Profit  margin  maximization. 

(1-10) 

5 

Accountability  of  all  the  participants. 

(1-10) 

6 

Transfer  of  risk  and  accountability  to  vendor. 

(1-10) 

7 

Minimization  of  surprises. 

(1-10) 

8 

Coordination  with  client. 

(1-10) 

9 

Total  Number  of  Pilot  Projects  Surveyed 

# 

Old  Process  - 

Before  Project 

New  Process  - 

Before  Project 

5 

8 

5 

9 

5 

8 

4 

7 

5 

8 

7 

8 

5 

9 

6 

8 

3 
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